1. Field of the Invention
The present invention relates to an image processing device, an imaging device and an image processing method, and particularly relates to an image processing device, an imaging device and an image processing method that make it possible to obtain a parallax resistant to noise and equivalent to a parallax to be perceived by humans, in the case of detecting a parallax for multiple viewpoint images that are imaged by pupil division.
2. Description of the Related Art
A stereoscopic imaging device including an imaging element that performs imaging by a pupil-division scheme is known. In the case of using such an imaging element with a pupil-division scheme, as a method for calculating a parallax by performing matching between left and right images, a method in which the positional deviation amount between blocks to minimize the sum of squares of the pixel value differences is regarded as a right solution and is output is well known. Thus, typically, the calculation of the parallax is performed using a method (correlation method) in which the parallax is calculated by the correlation based on the pixel values of the left-eye image and right-eye image.
For example, PTL 1 (Japanese Patent Application Laid-Open No. 2002-191060) discloses a phase difference type image sensor that includes a micro-lens whose upper part is opened and a micro-lens whose lower part is opened, and thereby, makes it possible to obtain a left-eye image and a right-eye image. Further, it is disclosed to perform the matching between the left-eye image and the right-eye image by a correlation method and determine the distance information.
Further, PTL 2 (National Publication of International Patent Application No. 2011-515045) discloses a phase difference type image sensor that disposes photoelectric conversion elements at the left side and the right side for one cylindrical micro-lens, and thereby, makes it possible to obtain a left-eye image and a right-eye image.
Further, PTL 3 (WO 2011/118089) discloses that a pupil-division imaging is performed so that multiple viewpoint images (a first image and a second image) are acquired, and when different image processes are respectively performed for the multiple viewpoint images, the image processes are performed such that the image quality difference between the multiple viewpoint images after the image processes is reduced.
Further, PTL 4 (WO 2011/118077) discloses that a pupil-division imaging is performed so that multiple viewpoint images (a first parallax image and a second parallax image) are acquired, a restoration filter corresponding to the defocus amount and image height for each position in the multiple viewpoint images is selected for each position, and the deconvolution is performed for each position in the multiple viewpoint images based on the selected restoration filter, so that the multiple viewpoint images are restored.